In [ ]:
 

1. Import Libraries/Dataset

a. Import the required libraries and the dataset

In [1]:
import tensorflow as tf\
    , glob, os, numpy as np, PIL, matplotlib.pyplot as plt

b. Check the GPU available

In [2]:
gpus = tf.config.experimental.list_physical_devices('GPU')
In [3]:
tf.zeros((1, 1)).gpu()
WARNING:tensorflow:From <ipython-input-3-1a50780ee4c5>:1: _EagerTensorBase.gpu (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.identity instead.
Out[3]:
<tf.Tensor: shape=(1, 1), dtype=float32, numpy=array([[0.]], dtype=float32)>
In [ ]:
 
In [ ]:
 

2. Data Visualization and augmentation

a. Plot at least two samples from each class of the dataset

In [4]:
for i, c in enumerate(glob.glob('data/imagenette2-160/train/**')):
    print(*['-']*20, f"\n-- Class : {os.path.basename(c)}")
    _, axs = plt.subplots(1, 2)
    ims = glob.glob(f'{c}/*.JPEG')
    for ii in range(2):
        im = PIL.Image.open(ims[ii])
        axs[ii].imshow(im)
        axs[ii].set_title(os.path.basename(c))
        axs[ii].axis("off")
    plt.show()
- - - - - - - - - - - - - - - - - - - - 
-- Class : n01440764
- - - - - - - - - - - - - - - - - - - - 
-- Class : n02102040
- - - - - - - - - - - - - - - - - - - - 
-- Class : n02979186
- - - - - - - - - - - - - - - - - - - - 
-- Class : n03000684
- - - - - - - - - - - - - - - - - - - - 
-- Class : n03028079
- - - - - - - - - - - - - - - - - - - - 
-- Class : n03394916
- - - - - - - - - - - - - - - - - - - - 
-- Class : n03417042
- - - - - - - - - - - - - - - - - - - - 
-- Class : n03425413
- - - - - - - - - - - - - - - - - - - - 
-- Class : n03445777
- - - - - - - - - - - - - - - - - - - - 
-- Class : n03888257
In [ ]:
 

b. Apply rotation and height shift augmentation (horizontal_flip, width_shift_range) to the dataset separately. Print the augmented image and the original image for each class and each augmentation.

In [19]:
for i, c in enumerate(glob.glob('data/imagenette2-160/train/**')):
    print(f"-- Class : {os.path.basename(c)}")
    im_size = 3
    cols = 4
    _, axs = plt.subplots(1, cols, figsize=(im_size*cols,im_size))
    im = glob.glob(f'{c}/*.JPEG')[0]
    arr = np.array(PIL.Image.open(im), dtype='ubyte')
    #
    axs[0].imshow(arr)
    axs[0].set_title("Original")
    #
    axs[1].imshow(tf.keras.preprocessing.image.random_shift(
        arr, 0.5, 0.5, 0, 1, 2, 
#         'reflect'
    ))
    axs[1].set_title("Hieght and Width Shift")
    #
    axs[2].imshow(tf.keras.preprocessing.image.random_rotation(
        arr, 360, 0, 1, 2, 
#         'reflect'
    ))
    axs[2].set_title("Rotation")
    #
    axs[3].imshow(tf.image.flip_left_right(arr).numpy())
    axs[3].set_title("Horizontal Flip")
    #
    [ax.axis("off") for ax in axs]
    plt.show()
    
-- Class : n01440764
-- Class : n02102040
-- Class : n02979186
-- Class : n03000684
-- Class : n03028079
-- Class : n03394916
-- Class : n03417042
-- Class : n03425413
-- Class : n03445777
-- Class : n03888257
In [ ]:
 

c. Bring the train and test data in the required format.

In [4]:
def norm(arr):
    "Convert and array with range 0,255 to -1,1"
    return (arr - 127)/128

def denorm(arr):
    "Convert and array with range -1,1 to 0,255"
    return (arr*128)+127

train_dl = tf.keras.preprocessing.image.ImageDataGenerator(
    rotation_range=360,
    width_shift_range=0.3,
    height_shift_range=0.3,
    horizontal_flip=True,
    vertical_flip=True,
    preprocessing_function=norm,
    # rescale=1./255.
    
).flow_from_directory(
    'data/imagenette2-160/train',
    target_size=(160, 160)
)
valid_dl = tf.keras.preprocessing.image.ImageDataGenerator(
    # rescale=1./255.,
    preprocessing_function=norm,
).flow_from_directory(
    'data/imagenette2-160/val',
    target_size=(160, 160)
)
Found 9469 images belonging to 10 classes.
Found 3925 images belonging to 10 classes.

d. Print the shapes of train and test data.

In [15]:
x_batch, y_batch = next(train_dl)
In [16]:
x_batch.shape
Out[16]:
(32, 256, 256, 3)

X batch shape explaination:

  • 32 is the batch_Size
  • 256, 256 is image height and width
  • 3 is channels of image
In [17]:
y_batch.shape
Out[17]:
(32, 10)

Y batch shape explaination:

  • 32 is the batch_Size
  • 10 is the number of classes as this is categorical data and y will be in format one hot encodings
In [ ]:
 

Plotting some samples from the dataloader

In [6]:
i = 0
cols=4
reverse_class_maping = {j:i for i,j in train_dl.class_indices.items()}
while True:
    if i>=x_batch.shape[0]:
        break
    _, axs = plt.subplots(1, cols, figsize=(3*cols, 3))
    for ii in range(cols):
        if i>=x_batch.shape[0]:
            break
        arr = denorm(x_batch[i]).astype('ubyte')
        axs[ii].imshow(arr)
        axs[ii].set_title(reverse_class_maping[y_batch[i].argmax()])
        axs[ii].axis("off")
        i+=1
    plt.show()
In [ ]:
 

3. Model Building

  • Sequential Model layers- Use AT LEAST 3 hidden layers with appropriate input for each. Choose the best number for hidden units and give reasons.

    We are counting one block of convolution, activation function and batch norm layer as a hidden layer. We would like to use a small model with 4 Hidden Layers. After the hidden layers we would like to flatten the output and apply a dense layer on averag pooled activation to reduce the dimensions of output to that of target predictions. We have not applied softmax activation to the top layer because we want to use logits directly in the loss function as they seem to be more stable numerically. Reference: a-guide-to-transfer-learning-with-keras-using-resnet50

  • Add L2 regularization to all the layers.

    Added

  • Add one layer of dropout at the appropriate position and give reasons.

    We will add the dropout layer before all dense layers as they are more prone to overfitting then the Convalution layers. No dropout layer has been applied after the final predictions.

  • Choose the appropriate activation function for all the layers.

    We would be using Relu for all the hidden layers and finally a softmax layer at the end. The softmax is used at the end (Called top but) to scale the sum of confidene of all classes to 100% or 1.0 in floating point number.

In [ ]:
 

a. Create a sequential model

In [8]:
from tensorflow.keras import layers
In [9]:
reg = tf.keras.regularizers.l2()
reg = None
model = tf.keras.Sequential([
  layers.Conv2D(16, 3, 2, padding='same', input_shape=x_batch.shape[1:], data_format="channels_last", kernel_regularizer=reg),
  layers.BatchNormalization(),
  layers.Activation("relu"),
  #
  layers.Conv2D(32, 3, 2, padding='same', kernel_regularizer=reg),
  layers.BatchNormalization(),
  layers.Activation("relu"),
  #
  layers.Conv2D(64, 3, 2, padding='same', kernel_regularizer=reg),
  layers.BatchNormalization(),
  layers.Activation("relu"),
  #
  layers.Conv2D(128, 3, 2, padding='same', kernel_regularizer=reg),
  layers.BatchNormalization(),
  layers.Activation("relu"),
  #
  layers.Flatten(),
  layers.Dropout(0.2),
  layers.Dense(128),
  layers.BatchNormalization(),
  layers.Activation("relu"),
  #
  layers.Dropout(0.4),
  layers.Dense(len(train_dl.class_indices), kernel_regularizer=reg)
])
In [12]:
model.summary()
Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d (Conv2D)              (None, 80, 80, 16)        448       
_________________________________________________________________
batch_normalization (BatchNo (None, 80, 80, 16)        64        
_________________________________________________________________
activation (Activation)      (None, 80, 80, 16)        0         
_________________________________________________________________
conv2d_1 (Conv2D)            (None, 40, 40, 32)        4640      
_________________________________________________________________
batch_normalization_1 (Batch (None, 40, 40, 32)        128       
_________________________________________________________________
activation_1 (Activation)    (None, 40, 40, 32)        0         
_________________________________________________________________
conv2d_2 (Conv2D)            (None, 20, 20, 64)        18496     
_________________________________________________________________
batch_normalization_2 (Batch (None, 20, 20, 64)        256       
_________________________________________________________________
activation_2 (Activation)    (None, 20, 20, 64)        0         
_________________________________________________________________
conv2d_3 (Conv2D)            (None, 10, 10, 128)       73856     
_________________________________________________________________
batch_normalization_3 (Batch (None, 10, 10, 128)       512       
_________________________________________________________________
activation_3 (Activation)    (None, 10, 10, 128)       0         
_________________________________________________________________
flatten (Flatten)            (None, 12800)             0         
_________________________________________________________________
dropout (Dropout)            (None, 12800)             0         
_________________________________________________________________
dense (Dense)                (None, 128)               1638528   
_________________________________________________________________
batch_normalization_4 (Batch (None, 128)               512       
_________________________________________________________________
activation_4 (Activation)    (None, 128)               0         
_________________________________________________________________
dropout_1 (Dropout)          (None, 128)               0         
_________________________________________________________________
dense_1 (Dense)              (None, 10)                1290      
=================================================================
Total params: 1,738,730
Trainable params: 1,737,994
Non-trainable params: 736
_________________________________________________________________
In [12]:
 

4. Model Compilation

In [13]:
model.compile(
    optimizer=tf.keras.optimizers.Adam(0.003),
    loss=tf.keras.losses.BinaryCrossentropy(True),
    metrics=['accuracy']
)
In [ ]:
 

5. Model Training

In [13]:
import time
start = time.time()
In [14]:
history = model.fit_generator(
    train_dl,
    epochs=50,
    validation_data=valid_dl
)
/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/engine/training.py:1844: UserWarning: `Model.fit_generator` is deprecated and will be removed in a future version. Please use `Model.fit`, which supports generators.
  warnings.warn('`Model.fit_generator` is deprecated and '
Epoch 1/50
296/296 [==============================] - 92s 200ms/step - loss: 0.3867 - accuracy: 0.2065 - val_loss: 0.2773 - val_accuracy: 0.3529
Epoch 2/50
296/296 [==============================] - 59s 198ms/step - loss: 0.2768 - accuracy: 0.3538 - val_loss: 0.2708 - val_accuracy: 0.3758
Epoch 3/50
296/296 [==============================] - 58s 196ms/step - loss: 0.2658 - accuracy: 0.3876 - val_loss: 0.2557 - val_accuracy: 0.4252
Epoch 4/50
296/296 [==============================] - 58s 195ms/step - loss: 0.2576 - accuracy: 0.4087 - val_loss: 0.2394 - val_accuracy: 0.4589
Epoch 5/50
296/296 [==============================] - 58s 195ms/step - loss: 0.2525 - accuracy: 0.4251 - val_loss: 0.2704 - val_accuracy: 0.3893
Epoch 6/50
296/296 [==============================] - 58s 195ms/step - loss: 0.2480 - accuracy: 0.4451 - val_loss: 0.2713 - val_accuracy: 0.4094
Epoch 7/50
296/296 [==============================] - 57s 194ms/step - loss: 0.2406 - accuracy: 0.4656 - val_loss: 0.2253 - val_accuracy: 0.4973
Epoch 8/50
296/296 [==============================] - 58s 194ms/step - loss: 0.2367 - accuracy: 0.4723 - val_loss: 0.2511 - val_accuracy: 0.4466
Epoch 9/50
296/296 [==============================] - 58s 196ms/step - loss: 0.2296 - accuracy: 0.4924 - val_loss: 0.2115 - val_accuracy: 0.5546
Epoch 10/50
296/296 [==============================] - 58s 197ms/step - loss: 0.2272 - accuracy: 0.5031 - val_loss: 0.2306 - val_accuracy: 0.4961
Epoch 11/50
296/296 [==============================] - 58s 196ms/step - loss: 0.2245 - accuracy: 0.5163 - val_loss: 0.2923 - val_accuracy: 0.3926
Epoch 12/50
296/296 [==============================] - 58s 195ms/step - loss: 0.2191 - accuracy: 0.5269 - val_loss: 0.2100 - val_accuracy: 0.5498
Epoch 13/50
296/296 [==============================] - 58s 195ms/step - loss: 0.2208 - accuracy: 0.5244 - val_loss: 0.2043 - val_accuracy: 0.5636
Epoch 14/50
296/296 [==============================] - 58s 195ms/step - loss: 0.2156 - accuracy: 0.5368 - val_loss: 0.2155 - val_accuracy: 0.5434
Epoch 15/50
296/296 [==============================] - 58s 196ms/step - loss: 0.2132 - accuracy: 0.5395 - val_loss: 0.2072 - val_accuracy: 0.5610
Epoch 16/50
296/296 [==============================] - 58s 197ms/step - loss: 0.2084 - accuracy: 0.5572 - val_loss: 0.2355 - val_accuracy: 0.5042
Epoch 17/50
296/296 [==============================] - 58s 196ms/step - loss: 0.2092 - accuracy: 0.5578 - val_loss: 0.2003 - val_accuracy: 0.5748
Epoch 18/50
296/296 [==============================] - 58s 196ms/step - loss: 0.2090 - accuracy: 0.5540 - val_loss: 0.2418 - val_accuracy: 0.4968
Epoch 19/50
296/296 [==============================] - 57s 194ms/step - loss: 0.2065 - accuracy: 0.5535 - val_loss: 0.2080 - val_accuracy: 0.5587
Epoch 20/50
296/296 [==============================] - 57s 193ms/step - loss: 0.2068 - accuracy: 0.5585 - val_loss: 0.2014 - val_accuracy: 0.5753
Epoch 21/50
296/296 [==============================] - 57s 193ms/step - loss: 0.2019 - accuracy: 0.5716 - val_loss: 0.2233 - val_accuracy: 0.5287
Epoch 22/50
296/296 [==============================] - 57s 193ms/step - loss: 0.1957 - accuracy: 0.5868 - val_loss: 0.1929 - val_accuracy: 0.5878
Epoch 23/50
296/296 [==============================] - 57s 192ms/step - loss: 0.1999 - accuracy: 0.5782 - val_loss: 0.2078 - val_accuracy: 0.5575
Epoch 24/50
296/296 [==============================] - 57s 193ms/step - loss: 0.1954 - accuracy: 0.5950 - val_loss: 0.2114 - val_accuracy: 0.5615
Epoch 25/50
296/296 [==============================] - 57s 193ms/step - loss: 0.1924 - accuracy: 0.5939 - val_loss: 0.1932 - val_accuracy: 0.6031
Epoch 26/50
296/296 [==============================] - 57s 193ms/step - loss: 0.1943 - accuracy: 0.5951 - val_loss: 0.2079 - val_accuracy: 0.5773
Epoch 27/50
296/296 [==============================] - 57s 194ms/step - loss: 0.1918 - accuracy: 0.6027 - val_loss: 0.1974 - val_accuracy: 0.5898
Epoch 28/50
296/296 [==============================] - 57s 194ms/step - loss: 0.1889 - accuracy: 0.6083 - val_loss: 0.2014 - val_accuracy: 0.5888
Epoch 29/50
296/296 [==============================] - 57s 193ms/step - loss: 0.1871 - accuracy: 0.6080 - val_loss: 0.1872 - val_accuracy: 0.6112
Epoch 30/50
296/296 [==============================] - 57s 192ms/step - loss: 0.1869 - accuracy: 0.6058 - val_loss: 0.1824 - val_accuracy: 0.6234
Epoch 31/50
296/296 [==============================] - 57s 192ms/step - loss: 0.1846 - accuracy: 0.6226 - val_loss: 0.1909 - val_accuracy: 0.5990
Epoch 32/50
296/296 [==============================] - 57s 194ms/step - loss: 0.1828 - accuracy: 0.6259 - val_loss: 0.1979 - val_accuracy: 0.5959
Epoch 33/50
296/296 [==============================] - 57s 192ms/step - loss: 0.1837 - accuracy: 0.6174 - val_loss: 0.1817 - val_accuracy: 0.6268
Epoch 34/50
296/296 [==============================] - 57s 193ms/step - loss: 0.1848 - accuracy: 0.6156 - val_loss: 0.1737 - val_accuracy: 0.6454
Epoch 35/50
296/296 [==============================] - 57s 192ms/step - loss: 0.1794 - accuracy: 0.6315 - val_loss: 0.1871 - val_accuracy: 0.6084
Epoch 36/50
296/296 [==============================] - 57s 192ms/step - loss: 0.1762 - accuracy: 0.6444 - val_loss: 0.1908 - val_accuracy: 0.6064
Epoch 37/50
296/296 [==============================] - 57s 193ms/step - loss: 0.1792 - accuracy: 0.6349 - val_loss: 0.1759 - val_accuracy: 0.6425
Epoch 38/50
296/296 [==============================] - 57s 192ms/step - loss: 0.1795 - accuracy: 0.6249 - val_loss: 0.1778 - val_accuracy: 0.6321
Epoch 39/50
296/296 [==============================] - 57s 193ms/step - loss: 0.1782 - accuracy: 0.6338 - val_loss: 0.1758 - val_accuracy: 0.6352
Epoch 40/50
296/296 [==============================] - 57s 193ms/step - loss: 0.1763 - accuracy: 0.6338 - val_loss: 0.1623 - val_accuracy: 0.6706
Epoch 41/50
296/296 [==============================] - 57s 192ms/step - loss: 0.1744 - accuracy: 0.6409 - val_loss: 0.1728 - val_accuracy: 0.6504
Epoch 42/50
296/296 [==============================] - 57s 192ms/step - loss: 0.1755 - accuracy: 0.6447 - val_loss: 0.2189 - val_accuracy: 0.5633
Epoch 43/50
296/296 [==============================] - 57s 193ms/step - loss: 0.1719 - accuracy: 0.6483 - val_loss: 0.1898 - val_accuracy: 0.6076
Epoch 44/50
296/296 [==============================] - 57s 192ms/step - loss: 0.1743 - accuracy: 0.6483 - val_loss: 0.1865 - val_accuracy: 0.6145
Epoch 45/50
296/296 [==============================] - 57s 192ms/step - loss: 0.1695 - accuracy: 0.6533 - val_loss: 0.1629 - val_accuracy: 0.6685
Epoch 46/50
296/296 [==============================] - 57s 192ms/step - loss: 0.1745 - accuracy: 0.6500 - val_loss: 0.1801 - val_accuracy: 0.6336
Epoch 47/50
296/296 [==============================] - 57s 192ms/step - loss: 0.1700 - accuracy: 0.6594 - val_loss: 0.1718 - val_accuracy: 0.6443
Epoch 48/50
296/296 [==============================] - 57s 192ms/step - loss: 0.1667 - accuracy: 0.6624 - val_loss: 0.1802 - val_accuracy: 0.6318
Epoch 49/50
296/296 [==============================] - 57s 192ms/step - loss: 0.1700 - accuracy: 0.6551 - val_loss: 0.1723 - val_accuracy: 0.6645
Epoch 50/50
296/296 [==============================] - 57s 192ms/step - loss: 0.1668 - accuracy: 0.6632 - val_loss: 0.1882 - val_accuracy: 0.6092
In [18]:
f"-- Total Time Taken to train model: {(time.time() - start)/60:0.3f} Minutes"
Out[18]:
'-- Total Time Taken to train model: 47.500 Minutes'
In [ ]:
 

Save model

In [16]:
model.save('v1_50e')
INFO:tensorflow:Assets written to: v1_50e/assets
In [ ]:
 

6. Model Evaluation

a. Print the final train and validation loss and accuracy

In [44]:
h = f"""
-- Final Model Metrics: 
> Training Loss: {history.history['loss'][-1]:0.3f}     | Validation Loss: {history.history['val_loss'][-1]:0.3f} 
> Training Accuracy: {history.history['accuracy'][-1]:0.3f} | Validation Accuracy: {history.history['val_accuracy'][-1]:0.3f} 
"""
print(h)
-- Final Model Metrics: 
> Training Loss: 0.166     | Validation Loss: 0.188 
> Training Accuracy: 0.666 | Validation Accuracy: 0.609 

In [49]:
 
In [48]:
#Confution Matrix and Classification Report
Y_pred = model.predict_generator(valid_dl)
y_pred = np.argmax(Y_pred, axis=1)
/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/engine/training.py:1905: UserWarning: `Model.predict_generator` is deprecated and will be removed in a future version. Please use `Model.predict`, which supports generators.
  warnings.warn('`Model.predict_generator` is deprecated and '
In [55]:
from sklearn.metrics import classification_report, confusion_matrix, plot_confusion_matrix
In [60]:
print('Confusion Matrix')
cm = confusion_matrix(valid_dl.classes, y_pred)
print(cm)
Confusion Matrix
[[255   1   4  25  60   4   3  11  18   6]
 [ 16 192  10  49  46   4   4  45  20   9]
 [  1   0 258   6  12   5   3  65   4   3]
 [  6   5  22 138  64  10   3 108  16  14]
 [  1   2  15   1 291  17   3  49   8  22]
 [  6   2  19   3  60 228   1  68   3   4]
 [  0   1  27  22  76   8 145 105   2   3]
 [  2   0  20  11  37   7   5 326   5   6]
 [ 13   5  13   7  39   2   2  35 257  26]
 [  7   0   5   9  13   4   9  29  13 301]]
In [59]:
import seaborn as sn
import pandas as pd
import matplotlib.pyplot as plt
In [94]:
df_cm = pd.DataFrame(cm, valid_dl.class_indices.keys(), valid_dl.class_indices.keys())
plt.figure(figsize=(18,14))
sn.set(font_scale=1.4) # for label size
sn.heatmap(df_cm, cmap='viridis', annot=True, annot_kws={"size": 16}) # font size

plt.show()

Classification Report

In [97]:
rep = classification_report(valid_dl.classes, y_pred, None, valid_dl.class_indices.keys())
print(rep)
              precision    recall  f1-score   support

   n01440764       0.83      0.66      0.73       387
   n02102040       0.92      0.49      0.64       395
   n02979186       0.66      0.72      0.69       357
   n03000684       0.51      0.36      0.42       386
   n03028079       0.42      0.71      0.53       409
   n03394916       0.79      0.58      0.67       394
   n03417042       0.81      0.37      0.51       389
   n03425413       0.39      0.78      0.52       419
   n03445777       0.74      0.64      0.69       399
   n03888257       0.76      0.77      0.77       390

    accuracy                           0.61      3925
   macro avg       0.68      0.61      0.62      3925
weighted avg       0.68      0.61      0.61      3925

If we look at the above classification report:

  • The best performing class is: n03888257
  • The worst performing class is n03000684
In [ ]:
 

b. Print the two most incorrectly classified images for each class in the test dataset.

In [132]:
conf_ground_truth = np.zeros((valid_dl.n, valid_dl.num_classes), 'float32')
conf_ground_truth[np.arange(valid_dl.n), valid_dl.classes] = 1.
conf_ground_truth
Out[132]:
array([[1., 0., 0., ..., 0., 0., 0.],
       [1., 0., 0., ..., 0., 0., 0.],
       [1., 0., 0., ..., 0., 0., 0.],
       ...,
       [0., 0., 0., ..., 0., 0., 1.],
       [0., 0., 0., ..., 0., 0., 1.],
       [0., 0., 0., ..., 0., 0., 1.]], dtype=float32)
In [140]:
def softmax(x: np.ndarray) -> np.ndarray:
    e_x = np.exp(x - np.max(x, axis=-1)[..., None])
    e_y = e_x.sum(axis=-1)[..., None]
    return e_x / e_y
In [188]:
bce = tf.keras.losses.BinaryCrossentropy(reduction=tf.keras.losses.Reduction.NONE)
Y_pred_sft = softmax(Y_pred)
losses = bce(conf_ground_truth,  Y_pred_sft).numpy()
argsrt_losses = losses.argsort()
In [256]:
for cn in valid_dl.class_indices:
    print(f"\n\n== Class : {cn}")
    cv = valid_dl.class_indices[cn]
    mask = valid_dl.classes[argsrt_losses] == cv
    for i, ii in enumerate(argsrt_losses[mask][:-3:-1]):
        print(f"-- Image: {valid_dl.filepaths[ii]}")
        _, axs = plt.subplots(1, 2, figsize=(12, 3))
        axs[0].imshow(PIL.Image.open(valid_dl.filepaths[ii]))
        axs[0].axis("off")
        # axs[0].set_title(valid_dl.filepaths[ii])
        #
        # axs[1].axis("off")
        c = ['b']*valid_dl.num_classes
        c[valid_dl.classes[ii]] = 'r'
        axs[1].bar(list(valid_dl.class_indices.keys()), Y_pred_sft[ii], 1, color=c)
        plt.xticks(rotation=90)
        plt.show()
        print("".join(["="]*25))
#

== Class : n01440764
-- Image: data/imagenette2-160/val/n01440764/n01440764_10462.JPEG
-- Image: data/imagenette2-160/val/n01440764/n01440764_10592.JPEG
=========================


== Class : n02102040
-- Image: data/imagenette2-160/val/n02102040/n02102040_1910.JPEG
-- Image: data/imagenette2-160/val/n02102040/n02102040_1781.JPEG
=========================


== Class : n02979186
-- Image: data/imagenette2-160/val/n02979186/n02979186_20561.JPEG
-- Image: data/imagenette2-160/val/n02979186/n02979186_6012.JPEG
=========================


== Class : n03000684
-- Image: data/imagenette2-160/val/n03000684/n03000684_23960.JPEG
-- Image: data/imagenette2-160/val/n03000684/n03000684_25951.JPEG
=========================


== Class : n03028079
-- Image: data/imagenette2-160/val/n03028079/n03028079_32202.JPEG
-- Image: data/imagenette2-160/val/n03028079/n03028079_8162.JPEG
=========================


== Class : n03394916
-- Image: data/imagenette2-160/val/n03394916/n03394916_42550.JPEG
-- Image: data/imagenette2-160/val/n03394916/n03394916_54531.JPEG
=========================


== Class : n03417042
-- Image: data/imagenette2-160/val/n03417042/n03417042_3710.JPEG
-- Image: data/imagenette2-160/val/n03417042/n03417042_3140.JPEG
=========================


== Class : n03425413
-- Image: data/imagenette2-160/val/n03425413/n03425413_14040.JPEG
-- Image: data/imagenette2-160/val/n03425413/n03425413_11061.JPEG
=========================


== Class : n03445777
-- Image: data/imagenette2-160/val/n03445777/n03445777_5932.JPEG
-- Image: data/imagenette2-160/val/n03445777/n03445777_5492.JPEG
=========================


== Class : n03888257
-- Image: data/imagenette2-160/val/n03888257/n03888257_21932.JPEG
-- Image: data/imagenette2-160/val/n03888257/n03888257_4732.JPEG
=========================
In [ ]:
 

7. Hyperparameter Tuning

7-1. Batch Size

Here we are doubling the batch size from the default size of 32 to 64. This would result in halving the steps per epoch as compared to batch size of 32.

In [22]:
train_dl.batch_size = 64
In [ ]:
reg = tf.keras.regularizers.l2()
reg = None
model = tf.keras.Sequential([
  layers.Conv2D(16, 3, 2, padding='same', input_shape=x_batch.shape[1:], data_format="channels_last", kernel_regularizer=reg),
  layers.BatchNormalization(),
  layers.Activation("relu"),
  #
  layers.Conv2D(32, 3, 2, padding='same', kernel_regularizer=reg),
  layers.BatchNormalization(),
  layers.Activation("relu"),
  #
  layers.Conv2D(64, 3, 2, padding='same', kernel_regularizer=reg),
  layers.BatchNormalization(),
  layers.Activation("relu"),
  #
  layers.Conv2D(128, 3, 2, padding='same', kernel_regularizer=reg),
  layers.BatchNormalization(),
  layers.Activation("relu"),
  #
  layers.Flatten(),
  layers.Dropout(0.2),
  layers.Dense(128),
  layers.BatchNormalization(),
  layers.Activation("relu"),
  #
  layers.Dropout(0.4),
  layers.Dense(len(train_dl.class_indices), kernel_regularizer=reg)
])
#
model.compile(
    optimizer=tf.keras.optimizers.Adam(0.003),
    loss=tf.keras.losses.BinaryCrossentropy(True),
    metrics=['accuracy']
)
In [262]:
import time
start = time.time()

history = model.fit_generator(
    train_dl,
    epochs=50,
    validation_data=valid_dl
)

f"-- Total Time Taken to train model: {(time.time() - start)/60:0.3f} Minutes"
/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/engine/training.py:1844: UserWarning: `Model.fit_generator` is deprecated and will be removed in a future version. Please use `Model.fit`, which supports generators.
  warnings.warn('`Model.fit_generator` is deprecated and '
Epoch 1/50
148/148 [==============================] - 57s 376ms/step - loss: 0.4217 - accuracy: 0.2225 - val_loss: 0.2936 - val_accuracy: 0.2991
Epoch 2/50
148/148 [==============================] - 55s 375ms/step - loss: 0.2849 - accuracy: 0.3369 - val_loss: 0.2606 - val_accuracy: 0.3926
Epoch 3/50
148/148 [==============================] - 55s 375ms/step - loss: 0.2680 - accuracy: 0.3857 - val_loss: 0.3716 - val_accuracy: 0.2617
Epoch 4/50
148/148 [==============================] - 56s 375ms/step - loss: 0.2570 - accuracy: 0.4149 - val_loss: 0.2436 - val_accuracy: 0.4446
Epoch 5/50
148/148 [==============================] - 56s 377ms/step - loss: 0.2519 - accuracy: 0.4304 - val_loss: 0.2857 - val_accuracy: 0.3878
Epoch 6/50
148/148 [==============================] - 55s 375ms/step - loss: 0.2422 - accuracy: 0.4616 - val_loss: 0.2531 - val_accuracy: 0.4354
Epoch 7/50
148/148 [==============================] - 56s 375ms/step - loss: 0.2392 - accuracy: 0.4762 - val_loss: 0.2236 - val_accuracy: 0.5083
Epoch 8/50
148/148 [==============================] - 56s 376ms/step - loss: 0.2343 - accuracy: 0.4868 - val_loss: 0.2347 - val_accuracy: 0.4927
Epoch 9/50
148/148 [==============================] - 56s 375ms/step - loss: 0.2294 - accuracy: 0.5034 - val_loss: 0.2360 - val_accuracy: 0.4882
Epoch 10/50
148/148 [==============================] - 55s 375ms/step - loss: 0.2262 - accuracy: 0.5059 - val_loss: 0.2357 - val_accuracy: 0.4856
Epoch 11/50
148/148 [==============================] - 56s 375ms/step - loss: 0.2247 - accuracy: 0.5132 - val_loss: 0.2260 - val_accuracy: 0.4981
Epoch 12/50
148/148 [==============================] - 56s 375ms/step - loss: 0.2191 - accuracy: 0.5248 - val_loss: 0.2142 - val_accuracy: 0.5340
Epoch 13/50
148/148 [==============================] - 56s 376ms/step - loss: 0.2154 - accuracy: 0.5427 - val_loss: 0.2022 - val_accuracy: 0.5605
Epoch 14/50
148/148 [==============================] - 55s 374ms/step - loss: 0.2133 - accuracy: 0.5400 - val_loss: 0.2150 - val_accuracy: 0.5274
Epoch 15/50
148/148 [==============================] - 55s 375ms/step - loss: 0.2121 - accuracy: 0.5478 - val_loss: 0.2066 - val_accuracy: 0.5554
Epoch 16/50
148/148 [==============================] - 56s 376ms/step - loss: 0.2062 - accuracy: 0.5572 - val_loss: 0.2118 - val_accuracy: 0.5419
Epoch 17/50
148/148 [==============================] - 55s 373ms/step - loss: 0.2049 - accuracy: 0.5597 - val_loss: 0.2045 - val_accuracy: 0.5608
Epoch 18/50
148/148 [==============================] - 56s 375ms/step - loss: 0.2059 - accuracy: 0.5579 - val_loss: 0.1963 - val_accuracy: 0.5842
Epoch 19/50
148/148 [==============================] - 55s 373ms/step - loss: 0.2041 - accuracy: 0.5635 - val_loss: 0.1993 - val_accuracy: 0.5717
Epoch 20/50
148/148 [==============================] - 56s 375ms/step - loss: 0.1970 - accuracy: 0.5925 - val_loss: 0.1901 - val_accuracy: 0.5977
Epoch 21/50
148/148 [==============================] - 56s 375ms/step - loss: 0.1987 - accuracy: 0.5887 - val_loss: 0.1970 - val_accuracy: 0.5982
Epoch 22/50
148/148 [==============================] - 55s 375ms/step - loss: 0.1973 - accuracy: 0.5867 - val_loss: 0.2097 - val_accuracy: 0.5625
Epoch 23/50
148/148 [==============================] - 55s 374ms/step - loss: 0.1924 - accuracy: 0.5935 - val_loss: 0.2199 - val_accuracy: 0.5363
Epoch 24/50
148/148 [==============================] - 55s 374ms/step - loss: 0.1909 - accuracy: 0.6102 - val_loss: 0.2079 - val_accuracy: 0.5740
Epoch 25/50
148/148 [==============================] - 55s 374ms/step - loss: 0.1904 - accuracy: 0.5983 - val_loss: 0.2190 - val_accuracy: 0.5315
Epoch 26/50
148/148 [==============================] - 56s 377ms/step - loss: 0.1933 - accuracy: 0.5975 - val_loss: 0.1746 - val_accuracy: 0.6415
Epoch 27/50
148/148 [==============================] - 56s 379ms/step - loss: 0.1898 - accuracy: 0.6042 - val_loss: 0.1840 - val_accuracy: 0.6140
Epoch 28/50
148/148 [==============================] - 56s 376ms/step - loss: 0.1887 - accuracy: 0.6035 - val_loss: 0.2095 - val_accuracy: 0.5850
Epoch 29/50
148/148 [==============================] - 56s 377ms/step - loss: 0.1825 - accuracy: 0.6226 - val_loss: 0.1800 - val_accuracy: 0.6367
Epoch 30/50
148/148 [==============================] - 56s 376ms/step - loss: 0.1834 - accuracy: 0.6177 - val_loss: 0.1893 - val_accuracy: 0.6110
Epoch 31/50
148/148 [==============================] - 56s 375ms/step - loss: 0.1821 - accuracy: 0.6245 - val_loss: 0.1735 - val_accuracy: 0.6484
Epoch 32/50
148/148 [==============================] - 55s 374ms/step - loss: 0.1785 - accuracy: 0.6353 - val_loss: 0.1787 - val_accuracy: 0.6268
Epoch 33/50
148/148 [==============================] - 56s 376ms/step - loss: 0.1834 - accuracy: 0.6261 - val_loss: 0.2184 - val_accuracy: 0.5610
Epoch 34/50
148/148 [==============================] - 55s 375ms/step - loss: 0.1769 - accuracy: 0.6375 - val_loss: 0.1870 - val_accuracy: 0.6079
Epoch 35/50
148/148 [==============================] - 55s 374ms/step - loss: 0.1769 - accuracy: 0.6345 - val_loss: 0.1718 - val_accuracy: 0.6433
Epoch 36/50
148/148 [==============================] - 56s 376ms/step - loss: 0.1750 - accuracy: 0.6472 - val_loss: 0.2079 - val_accuracy: 0.5738
Epoch 37/50
148/148 [==============================] - 56s 376ms/step - loss: 0.1708 - accuracy: 0.6557 - val_loss: 0.1671 - val_accuracy: 0.6606
Epoch 38/50
148/148 [==============================] - 56s 376ms/step - loss: 0.1724 - accuracy: 0.6418 - val_loss: 0.2354 - val_accuracy: 0.5531
Epoch 39/50
148/148 [==============================] - 55s 375ms/step - loss: 0.1712 - accuracy: 0.6472 - val_loss: 0.1969 - val_accuracy: 0.6036
Epoch 40/50
148/148 [==============================] - 56s 375ms/step - loss: 0.1725 - accuracy: 0.6489 - val_loss: 0.1768 - val_accuracy: 0.6410
Epoch 41/50
148/148 [==============================] - 55s 374ms/step - loss: 0.1740 - accuracy: 0.6490 - val_loss: 0.1677 - val_accuracy: 0.6629
Epoch 42/50
148/148 [==============================] - 56s 376ms/step - loss: 0.1686 - accuracy: 0.6587 - val_loss: 0.1726 - val_accuracy: 0.6484
Epoch 43/50
148/148 [==============================] - 55s 375ms/step - loss: 0.1687 - accuracy: 0.6557 - val_loss: 0.2105 - val_accuracy: 0.5819
Epoch 44/50
148/148 [==============================] - 55s 374ms/step - loss: 0.1704 - accuracy: 0.6509 - val_loss: 0.1788 - val_accuracy: 0.6359
Epoch 45/50
148/148 [==============================] - 55s 374ms/step - loss: 0.1686 - accuracy: 0.6549 - val_loss: 0.1883 - val_accuracy: 0.6283
Epoch 46/50
148/148 [==============================] - 55s 374ms/step - loss: 0.1662 - accuracy: 0.6660 - val_loss: 0.1887 - val_accuracy: 0.6163
Epoch 47/50
148/148 [==============================] - 55s 375ms/step - loss: 0.1695 - accuracy: 0.6526 - val_loss: 0.1643 - val_accuracy: 0.6739
Epoch 48/50
148/148 [==============================] - 55s 374ms/step - loss: 0.1629 - accuracy: 0.6666 - val_loss: 0.1895 - val_accuracy: 0.6359
Epoch 49/50
148/148 [==============================] - 56s 376ms/step - loss: 0.1594 - accuracy: 0.6853 - val_loss: 0.1861 - val_accuracy: 0.6214
Epoch 50/50
148/148 [==============================] - 55s 374ms/step - loss: 0.1627 - accuracy: 0.6604 - val_loss: 0.1834 - val_accuracy: 0.6265
Out[262]:
'-- Total Time Taken to train model: 46.301 Minutes'
In [3]:
h = f"""
-- Final Model Metrics: 
> Training Loss: {history.history['loss'][-1]:0.3f}     | Validation Loss: {history.history['val_loss'][-1]:0.3f} 
> Training Accuracy: {history.history['accuracy'][-1]:0.3f} | Validation Accuracy: {history.history['val_accuracy'][-1]:0.3f} 
"""
print(h)
-- Final Model Metrics: 
> Training Loss: 0.163     | Validation Loss: 0.183 
> Training Accuracy: 0.660 | Validation Accuracy: 0.626 

We have changed the batch size to double which will initially halve the number of steps and double the loss magnitude. The loss magnitude has doubles because of sum reduction of loss calculated over each image in the batch.

In [ ]:
 

7-2. Dropout

In [264]:
## Reset Batch size
train_dl.batch_size = 32
In [263]:
reg = tf.keras.regularizers.l2()
reg = None
model = tf.keras.Sequential([
  layers.Conv2D(16, 3, 2, padding='same', input_shape=x_batch.shape[1:], data_format="channels_last", kernel_regularizer=reg),
  layers.BatchNormalization(),
  layers.Activation("relu"),
  #
  layers.Conv2D(32, 3, 2, padding='same', kernel_regularizer=reg),
  layers.BatchNormalization(),
  layers.Activation("relu"),
  #
  layers.Conv2D(64, 3, 2, padding='same', kernel_regularizer=reg),
  layers.BatchNormalization(),
  layers.Activation("relu"),
  #
  layers.Dropout(0.4),
  layers.Conv2D(128, 3, 2, padding='same', kernel_regularizer=reg),
  layers.BatchNormalization(),
  layers.Activation("relu"),
  #
  layers.Flatten(),
  layers.Dense(128),
  layers.BatchNormalization(),
  layers.Activation("relu"),
  #
  layers.Dropout(0.5),
  layers.Dense(len(train_dl.class_indices), kernel_regularizer=reg)
])
#
model.compile(
    optimizer=tf.keras.optimizers.Adam(0.003),
    loss=tf.keras.losses.BinaryCrossentropy(True),
    metrics=['accuracy']
)
In [265]:
import time
start = time.time()

history = model.fit_generator(
    train_dl,
    epochs=50,
    validation_data=valid_dl
)

f"-- Total Time Taken to train model: {(time.time() - start)/60:0.3f} Minutes"
/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/engine/training.py:1844: UserWarning: `Model.fit_generator` is deprecated and will be removed in a future version. Please use `Model.fit`, which supports generators.
  warnings.warn('`Model.fit_generator` is deprecated and '
Epoch 1/50
296/296 [==============================] - 58s 191ms/step - loss: 0.4048 - accuracy: 0.1952 - val_loss: 0.2911 - val_accuracy: 0.2994
Epoch 2/50
296/296 [==============================] - 57s 191ms/step - loss: 0.2879 - accuracy: 0.3188 - val_loss: 0.2653 - val_accuracy: 0.3738
Epoch 3/50
296/296 [==============================] - 56s 190ms/step - loss: 0.2698 - accuracy: 0.3800 - val_loss: 0.3705 - val_accuracy: 0.2645
Epoch 4/50
296/296 [==============================] - 57s 191ms/step - loss: 0.2619 - accuracy: 0.3998 - val_loss: 0.2483 - val_accuracy: 0.4201
Epoch 5/50
296/296 [==============================] - 57s 191ms/step - loss: 0.2567 - accuracy: 0.4299 - val_loss: 0.2531 - val_accuracy: 0.4257
Epoch 6/50
296/296 [==============================] - 56s 191ms/step - loss: 0.2539 - accuracy: 0.4283 - val_loss: 0.2329 - val_accuracy: 0.4634
Epoch 7/50
296/296 [==============================] - 56s 190ms/step - loss: 0.2483 - accuracy: 0.4443 - val_loss: 0.2208 - val_accuracy: 0.5159
Epoch 8/50
296/296 [==============================] - 57s 191ms/step - loss: 0.2437 - accuracy: 0.4576 - val_loss: 0.2197 - val_accuracy: 0.5233
Epoch 9/50
296/296 [==============================] - 57s 192ms/step - loss: 0.2398 - accuracy: 0.4669 - val_loss: 0.2155 - val_accuracy: 0.5141
Epoch 10/50
296/296 [==============================] - 56s 191ms/step - loss: 0.2379 - accuracy: 0.4720 - val_loss: 0.2254 - val_accuracy: 0.4907
Epoch 11/50
296/296 [==============================] - 57s 192ms/step - loss: 0.2330 - accuracy: 0.4939 - val_loss: 0.2207 - val_accuracy: 0.5203
Epoch 12/50
296/296 [==============================] - 56s 191ms/step - loss: 0.2314 - accuracy: 0.4914 - val_loss: 0.2107 - val_accuracy: 0.5541
Epoch 13/50
296/296 [==============================] - 56s 190ms/step - loss: 0.2234 - accuracy: 0.5190 - val_loss: 0.2176 - val_accuracy: 0.5322
Epoch 14/50
296/296 [==============================] - 56s 190ms/step - loss: 0.2186 - accuracy: 0.5328 - val_loss: 0.2031 - val_accuracy: 0.5592
Epoch 15/50
296/296 [==============================] - 56s 191ms/step - loss: 0.2188 - accuracy: 0.5290 - val_loss: 0.2316 - val_accuracy: 0.4976
Epoch 16/50
296/296 [==============================] - 57s 192ms/step - loss: 0.2208 - accuracy: 0.5258 - val_loss: 0.2095 - val_accuracy: 0.5511
Epoch 17/50
296/296 [==============================] - 57s 192ms/step - loss: 0.2141 - accuracy: 0.5457 - val_loss: 0.2169 - val_accuracy: 0.5213
Epoch 18/50
296/296 [==============================] - 56s 191ms/step - loss: 0.2160 - accuracy: 0.5307 - val_loss: 0.2175 - val_accuracy: 0.5404
Epoch 19/50
296/296 [==============================] - 57s 191ms/step - loss: 0.2127 - accuracy: 0.5478 - val_loss: 0.1941 - val_accuracy: 0.5913
Epoch 20/50
296/296 [==============================] - 56s 190ms/step - loss: 0.2084 - accuracy: 0.5587 - val_loss: 0.2020 - val_accuracy: 0.5679
Epoch 21/50
296/296 [==============================] - 57s 191ms/step - loss: 0.2089 - accuracy: 0.5660 - val_loss: 0.2546 - val_accuracy: 0.4932
Epoch 22/50
296/296 [==============================] - 57s 191ms/step - loss: 0.2052 - accuracy: 0.5677 - val_loss: 0.2246 - val_accuracy: 0.5228
Epoch 23/50
296/296 [==============================] - 56s 191ms/step - loss: 0.2048 - accuracy: 0.5662 - val_loss: 0.1860 - val_accuracy: 0.6110
Epoch 24/50
296/296 [==============================] - 56s 191ms/step - loss: 0.2062 - accuracy: 0.5587 - val_loss: 0.2162 - val_accuracy: 0.5468
Epoch 25/50
296/296 [==============================] - 56s 190ms/step - loss: 0.2037 - accuracy: 0.5758 - val_loss: 0.2002 - val_accuracy: 0.5827
Epoch 26/50
296/296 [==============================] - 56s 191ms/step - loss: 0.1988 - accuracy: 0.5858 - val_loss: 0.1884 - val_accuracy: 0.6028
Epoch 27/50
296/296 [==============================] - 56s 191ms/step - loss: 0.1984 - accuracy: 0.5846 - val_loss: 0.1896 - val_accuracy: 0.6036
Epoch 28/50
296/296 [==============================] - 56s 191ms/step - loss: 0.2009 - accuracy: 0.5755 - val_loss: 0.1952 - val_accuracy: 0.5944
Epoch 29/50
296/296 [==============================] - 56s 190ms/step - loss: 0.1981 - accuracy: 0.5846 - val_loss: 0.2060 - val_accuracy: 0.5750
Epoch 30/50
296/296 [==============================] - 56s 191ms/step - loss: 0.1962 - accuracy: 0.5803 - val_loss: 0.1936 - val_accuracy: 0.6117
Epoch 31/50
296/296 [==============================] - 57s 192ms/step - loss: 0.1974 - accuracy: 0.5899 - val_loss: 0.1952 - val_accuracy: 0.5873
Epoch 32/50
296/296 [==============================] - 57s 193ms/step - loss: 0.1923 - accuracy: 0.6071 - val_loss: 0.1905 - val_accuracy: 0.6071
Epoch 33/50
296/296 [==============================] - 57s 192ms/step - loss: 0.1944 - accuracy: 0.5917 - val_loss: 0.1876 - val_accuracy: 0.6033
Epoch 34/50
296/296 [==============================] - 57s 192ms/step - loss: 0.1888 - accuracy: 0.6120 - val_loss: 0.1681 - val_accuracy: 0.6586
Epoch 35/50
296/296 [==============================] - 56s 191ms/step - loss: 0.1907 - accuracy: 0.6073 - val_loss: 0.1791 - val_accuracy: 0.6352
Epoch 36/50
296/296 [==============================] - 57s 191ms/step - loss: 0.1926 - accuracy: 0.6091 - val_loss: 0.1926 - val_accuracy: 0.6087
Epoch 37/50
296/296 [==============================] - 57s 191ms/step - loss: 0.1854 - accuracy: 0.6166 - val_loss: 0.1768 - val_accuracy: 0.6408
Epoch 38/50
296/296 [==============================] - 57s 193ms/step - loss: 0.1871 - accuracy: 0.6139 - val_loss: 0.1980 - val_accuracy: 0.5929
Epoch 39/50
296/296 [==============================] - 57s 192ms/step - loss: 0.1881 - accuracy: 0.6120 - val_loss: 0.2017 - val_accuracy: 0.5913
Epoch 40/50
296/296 [==============================] - 56s 191ms/step - loss: 0.1836 - accuracy: 0.6282 - val_loss: 0.1813 - val_accuracy: 0.6308
Epoch 41/50
296/296 [==============================] - 57s 192ms/step - loss: 0.1873 - accuracy: 0.6125 - val_loss: 0.1694 - val_accuracy: 0.6594
Epoch 42/50
296/296 [==============================] - 57s 191ms/step - loss: 0.1854 - accuracy: 0.6148 - val_loss: 0.1861 - val_accuracy: 0.6171
Epoch 43/50
296/296 [==============================] - 56s 191ms/step - loss: 0.1835 - accuracy: 0.6273 - val_loss: 0.2292 - val_accuracy: 0.5294
Epoch 44/50
296/296 [==============================] - 56s 191ms/step - loss: 0.1827 - accuracy: 0.6269 - val_loss: 0.1678 - val_accuracy: 0.6660
Epoch 45/50
296/296 [==============================] - 57s 191ms/step - loss: 0.1823 - accuracy: 0.6250 - val_loss: 0.1891 - val_accuracy: 0.6166
Epoch 46/50
296/296 [==============================] - 57s 191ms/step - loss: 0.1817 - accuracy: 0.6248 - val_loss: 0.1614 - val_accuracy: 0.6772
Epoch 47/50
296/296 [==============================] - 57s 191ms/step - loss: 0.1841 - accuracy: 0.6167 - val_loss: 0.1851 - val_accuracy: 0.6293
Epoch 48/50
296/296 [==============================] - 57s 191ms/step - loss: 0.1802 - accuracy: 0.6327 - val_loss: 0.1732 - val_accuracy: 0.6510
Epoch 49/50
296/296 [==============================] - 57s 191ms/step - loss: 0.1795 - accuracy: 0.6370 - val_loss: 0.2028 - val_accuracy: 0.6099
Epoch 50/50
296/296 [==============================] - 56s 189ms/step - loss: 0.1807 - accuracy: 0.6346 - val_loss: 0.1789 - val_accuracy: 0.6290
Out[265]:
'-- Total Time Taken to train model: 47.161 Minutes'
In [4]:
h = f"""
-- Final Model Metrics: 
> Training Loss: {history.history['loss'][-1]:0.3f}     | Validation Loss: {history.history['val_loss'][-1]:0.3f} 
> Training Accuracy: {history.history['accuracy'][-1]:0.3f} | Validation Accuracy: {history.history['val_accuracy'][-1]:0.3f} 
"""
print(h)
-- Final Model Metrics: 
> Training Loss: 0.181     | Validation Loss: 0.179 
> Training Accuracy: 0.635 | Validation Accuracy: 0.629 

Here we increased the dropout a little bit before top layer and moved a dropout layer from between two dense layers to two convolution layers.

  • Because of the increasing the dropout before the top layer we see better validation loss and accuracy as compared to the previous model.
  • Because of moving a dropout layer from between two dense layers to between two convolutions layers we see a slight dip in the training accuracy.
In [ ]:
 
In [ ]:
 
In [ ]: